Work system and information processing method

ABSTRACT

In a work system which measures 3D information of a target object by a light-section method, high measurement precision is realized. The invention includes a robot arm, a hand position detection member which is arranged at the distal end portion of the robot arm, a slit laser projector which irradiates slit light, a camera which is fixed at a position independent of the robot arm, and senses an image of the target object, and a computer which calculates a light-section plane of the slit light based on a light-section line formed on the hand position detection member which is included in image data of the target object obtained by image sensing by the camera, and calculates the position and orientation of the target object based on the calculated light-section plane and the light-section line formed on the target object.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a work system which includes an armmechanism used to execute a predetermined work for a target object, andan information processing method in the work system.

2. Description of the Related Art

In a work system which includes an arm mechanism used to execute apredetermined work for a target object whose position and orientationchange, the position and orientation (three-dimensional (3D)information) of the target object are required to be measured uponexecution of the work, and a light-section method is known as themeasuring method.

The light-section method is a method that acquires 3D information of thetarget object by executing a triangulation by combining a camera andslit light projector, and various proposals about this method have beenconventionally made in association with application to the work system.

For example, Japanese Patent Laid-Open No. 2005-271103 proposes aposition calibration method of an origin when a 3D scanner (imagesensing device and slit light projector) is mounted on a robot arm.Also, Japanese Patent Laid-Open No. 2005-163346 discloses a method offocusing emitted beam light on a target object when a 3D scanner (imagesensing device and slit light projector) is mounted on a robot arm.

However, both the work systems disclosed in these patent references arepremised on the arrangement which integrates the image sensing deviceand slit light projector. For this reason, if the image sensing deviceis arranged at a global position (a fixed position not on the robotarm), the slit light projector is also arranged at the global position.When the image sensing device is arranged on the robot arm, the slitlight projector is also fixed on the robot arm.

However, when the image sensing device and slit light projector arearranged at the global position, occlusion of slit light emitted by theslit light projector by the target object itself or the robot arm cannotbe avoided depending on the position or orientation of the targetobject. Also, the focal depth of the emitted slit light often limit aprecise measurement range. These problems disturb achievement of highmeasurement precision.

On the other hand, when the image sensing device and slit lightprojector are arranged on the robot arm, they cannot be arranged so asto be spaced apart from each other. Since the light-section methodmeasures based on the principle of triangulation, when the distancebetween the image sensing device and slit light projector is short, andan angle the optical axis of the image sensing device makes with that ofthe slit light projector (optic angle) is small, the measurementprecision in the depth direction is decreased.

When the robot arm has a small size, it is difficult to arrange theimage sensing device which has a high resolution and a large weight. Inthis case, high measurement precision cannot be expected.

SUMMARY OF THE INVENTION

The present invention has been made in consideration of theaforementioned problems.

A work system according to the present invention comprises the followingarrangement. That is, a work system comprising: an arm mechanism, aposition and an orientation of which are configured to be changed so asto execute a predetermined work for an object to be processed placed ona work area; a member arranged at a distal end portion of the armmechanism; an irradiation unit configured to form a light projectionpattern on the member and on the object to be processed by irradiatingthe member and the object to be processed with pattern light, theirradiation unit being arranged on the arm mechanism to have a constantrelative position with respect to the member; an image sensing unitconfigured to sense an image of the object to be processed, the imagesensing unit being fixed at a position independent of the arm mechanism;a first calculation unit configured to calculate a position and anorientation of the distal end portion of the arm mechanism and anirradiation plane of the pattern light based on the light projectionpattern formed on the member included in image data of the object to beprocessed which is obtained by image sensing by the image sensing unit;and a second calculation unit configured to calculate a position and anorientation of the object to be processed based on the calculatedirradiation plane and the light projection pattern formed on the objectto be processed included in the image data of the object to be processedwhich is obtained by image sensing by the image sensing unit.

According to the present invention, in a work system which comprises anarm mechanism used to execute a predetermined work while measuringthree-dimensional information of a target object by the light-sectionmethod with respect to the target object whose position and orientationchange, high measurement precision can be assured.

Furthermore, a three-dimensional information measurement method andcalibration method based on the light-section method, which are appliedto the work system, can be provided.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of the specification, illustrate embodiments of the invention and,together with the description, serve to explain the principles of theinvention.

FIG. 1 is a view showing the outer appearance arrangement of a worksystem according to the first embodiment of the present invention;

FIG. 2 is a view showing the outer appearance arrangement of a workunit;

FIG. 3 is a block diagram showing the functional arrangement of a worksystem;

FIG. 4 is a view for explaining a calibration jig used to calibrate thework system;

FIG. 5 is a flowchart showing the sequence of calibration processing inthe work system;

FIG. 6 is a view showing an example of image data obtained by sensing animage of the calibration jig;

FIG. 7 is a view showing an example of image data obtained by sensing animage of the calibration jig;

FIG. 8 is a flowchart showing the sequence of work processing in thework system;

FIG. 9 is a flowchart showing the sequence of work processing in a worksystem according to the second embodiment of the present invention;

FIG. 10 is a view showing the outer appearance arrangement of a workunit of a work system according to the third embodiment of the presentinvention;

FIG. 11 is a flowchart showing the sequence of calibration processing inthe work system;

FIG. 12 is a view showing an example of light-section lines;

FIG. 13 is a flowchart showing the sequence of work processing in thework system;

FIG. 14 is a view showing the outer appearance arrangement of a workunit of a work system according to the fourth embodiment of the presentinvention;

FIG. 15 is a view showing an example of a sensed image using binarypattern light projected by a pattern light projector; and

FIG. 16 is a view showing examples of light projection patternsprojected by the pattern light projector.

DESCRIPTION OF THE EMBODIMENTS

Embodiments of the present invention will now be described in detail inaccordance with the accompanying drawings.

First Embodiment

<1. Outer Appearance Arrangement of Work System>

FIG. 1 is a view showing the outer appearance arrangement of a worksystem 100 according to the first embodiment of the present invention.Referring to FIG. 1, reference numeral 110 denotes a work unit whichincludes an arm mechanism used to execute a predetermined work for anobject to be processed (target object). Reference numeral 111 denotes arobot arm. A robot hand 112 as a member (end effecter) which contactsthe target object and executes the predetermined work is attached to thedistal end portion of the robot arm 111. Note that the robot arm 111 androbot hand 112 will be collectively referred to as an arm mechanismhereinafter.

Reference numeral 113 denotes a slit laser projector (slit lightprojector) which serves as a light projection unit. The slit laserprojector 113 is attached to the distal end portion of the robot arm111, and emits laser slit light as pattern light.

Reference numeral 114 denotes a hand position detection member whichserves as a detection member. The hand position detection member 114 isattached to be parallel to the robot hand 112 so that a relativeposition with respect to the slit laser projector 113 remains unchanged(to be constant).

The robot arm 111, robot hand 112, slit laser projector 113, and handposition detection member 114 configure the work unit 110.

Reference numeral 120 denotes a camera which serves as an image sensingunit. The camera 120 is arranged independently of the robot arm 111 soas to sense an image in a work area where the predetermined work isexecuted over the robot arm 111.

Reference numeral 130 denotes a robot controller which serves as a workcontrol unit. The robot arm 111 is connected to the robot controller130, and the robot arm 111 and robot hand 112 are controlled by therobot controller 130.

Reference numeral 140 denotes a computer which serves as a calculationunit. The computer 140 executes information processing required tocontrol the robot arm 111 and robot hand 112 via the robot controller130. Also, the computer 140 calculates the position and orientation (3Dinformation) of the target object based on image data sensed by thecamera 120. Reference numeral 150 denotes a work area where a targetobject 160 to be worked by the work unit 110 is placed.

As described above, the work system 100 according to this embodimentadopts the arrangement in which the image sensing unit 120 is arrangedat a position independent of the robot arm 111 without integrating thecamera 120, slit laser projector 113, and hand position detection member114. Also, the work system 100 adopts the arrangement in which the slitlaser projector 113 and hand position detection member 114 are arrangedon the robot arm 111.

With this arrangement, problems related to occlusion and that of thefocal depth of beam light can be avoided. Since a large optic angle canbe assured, high measurement precision can be attained even in the depthdirection, and the camera with a high resolution (i.e., with a largeweight) can be arranged irrespective of the size of the robot arm.

<2. Outer Appearance Arrangement of Work Unit>

The outer appearance arrangement of the work unit will be describedbelow. FIG. 2 is a view showing the outer appearance arrangement of thework unit 110.

Referring to FIG. 2, reference numeral 200 denotes a light-section planeformed by slit light emitted by the slit laser projector 113. If anobject exists at a position that intersects with the light-section plane200, a light-section line is generated at a position where that objectand the light-section plane intersect with each other.

Reference numeral 201 denotes a light-section line which is generatedwhen the light-section plane 200 of the slit light emitted by the slitlaser projector 113 intersects with the hand position detection member114.

Assume that the slit laser projector 113 is arranged at a position andangle which allow the light-section plane 200 of the emitted slit lightto generate the light-section line 201 on the hand position detectionmember 114.

Also, assume that the hand position detection member 114 has a shapewhose position and orientation are uniquely determined based on thelight-section line 201. More specifically, assume that the section ofthe hand position detection member 114 has a triangular shape, so thatthe light-section line 201 is bent halfway (i.e., it is defined by twoor more straight lines).

<3. Functional Arrangement of Work System>

The functional arrangement of the work system will be described below.FIG. 3 is a block diagram showing the functional arrangement of the worksystem 100.

In FIG. 3, since the work unit 110, image sensing unit (camera) 120, andwork control unit (robot controller) 130 have already been describedusing FIGS. 1 and 2, a detailed description thereof will not berepeated.

The calculation unit (computer) 140 includes a CPU, RAM, ROM, andstorage device (not shown), and executes various functions when the CPUexecutes programs loaded into the RAM. More specifically, thecalculation unit 140 serves as a measurement control unit 301,measurement value calculation unit 302, image data storage unit 303,calibration value calculation unit 304, and calibration value storageunit 305. Note that details of the respective functions will bedescribed later.

<4. Calibration Method Required to Measure 3D Information byLight-Section Method>

The calibration method required to measure the 3D information by thelight-section method will be described below.

<4.1 Description of Calibration Jig>

A calibration jig (a target object having a given position andorientation) used to calibrate the work system 100 will be describedfirst. FIG. 4 is a view for explaining a calibration jig used tocalibrate the work system 100.

Referring to FIG. 4, reference numeral 401 denotes a calibration base,which is placed on the work area 150 in place of the target object uponcalibration.

Note that the height of the calibration base 401 roughly matches that ofthe target object, but they need not strictly match.

Reference numeral 402 denotes a calibration jig, which is placed on thecalibration base 401. Assume that the calibration jig 402 has at leastsix vertices having given dimensions, and is defined by a plurality ofplanes. Also, assume that the relative positional relationship among thevertices is given with high precision.

In the example of FIG. 4, an object of a regular hexagonal pyramid isprepared as the calibration jig 402. However, the present invention isnot limited to such specific shape, and other shapes (for example, apentagonal pyramid, octagonal pyramid, etc.) may be used. A case will beexplained below wherein the calibration jig 402 has a shape of a regularhexagonal pyramid.

<4.2 Overview of Calibration Processing>

The sequence of the calibration processing in the work system will bedescribed below. FIG. 5 is a flowchart showing the sequence of thecalibration processing in the work system 100.

In step S501 (jig coordinate system setting process), the calibrationvalue calculation unit 304 sends an image sensing instruction to themeasurement control unit 301, which sends an image sensing controlsignal to the image sensing unit 120.

The image sensing unit 120 senses an image in response to the controlsignal from the measurement control unit 301, and stores acquired imagedata via the image data storage unit 303.

The calibration value calculation unit 304 sets a jig coordinate systembased on the image data stored via the image data storage unit 303, andconverts the jig coordinate system into an image coordinate system.

In step S502 (calibration movement process), the calibration valuecalculation unit 304 sends a moving instruction to a predeterminedposition to the work control unit 130, which sends a control signalbased on that instruction to the work unit 110. The work unit 110 movesto the predetermined position based on the control signal.

In step S503 (calibration data acquisition process), the calibrationvalue calculation unit 304 sends an image sensing start instruction andslit light irradiation start instruction to the measurement control unit301, which sends control signals to the image sensing unit 120 and lightprojection unit 113. The image sensing unit 120 starts image sensing andthe light projection unit 113 starts emission of slit light in responseto the control signals from the measurement control unit 301.

In step S504 (slit light state calculation process), the calibrationvalue calculation unit 304 detects a light-section line on thecalibration jig 402 in the image data using the image data stored viathe image data storage unit 303. Furthermore, the calibration valuecalculation unit 304 estimates a state of the slit light on the jigcoordinate system by making predetermined calculations with respect tothe position of the detected light-section line, and associates thestate of the work unit 110 with that of the slit light.

In step S505 (work unit state calculation process), the calibrationvalue calculation unit 304 detects a light-section line on the detectionmember 114 in the image data using the image data stored via the imagedata storage unit 303. Furthermore, the calibration value calculationunit 304 estimates a state of the work unit 110 on the jig coordinatesystem based on the state of the slit light obtained in the slit lightstate calculation process (step S504) with respect to the position ofthe detected light-section line, and associates the jig coordinatesystem of the work unit 110 with a work instruction to the work unit110.

It is determined in step S506 if image sensing processes of thepredetermined number of times are complete. If it is determined thatimage sensing processes of the predetermined number of times are notcomplete yet, the process returns to step S502 (calibration movementprocess). On the other hand, if it is determined that the image sensingprocesses of the predetermined number of times are complete, the processadvances to step S507 (calibration value calculation process).

In step S507 (calibration value calculation process), a correspondencefunction required to calculate the state of the slit light with respectto an arbitrary state of the work unit 110 is generated based on thecorresponding values obtained in the slit light state calculationprocess (step S504) and the work unit state calculation process (stepS505). Also, a correspondence function required to calculate the stateof the work unit 110 on a robot coordinate system from the state of thework unit 110 with respect to an arbitrary jig coordinate system isgenerated. Furthermore, parameters which configure the generatedcorrespondence functions are stored as calibration parameters via thecalibration value storage unit 305.

<4.3 Details of Calibration Processing>

Details of the calibration processing in FIG. 5 will be described below.

In step S501 (jig coordinate system setting process), image dataobtained by sensing an image of the calibration jig 402 using the camera120 is acquired (FIG. 6). Then, the vertices of the calibration jig 402in the acquired image data are detected by predetermined imageprocessing.

For instance, an equation of straight line is calculated by, forexample, the Hough transform from the edge extraction results ofrespective sides, an intersection of six straight lines is defined as atop vertex 600, and each intersection of three straight lines is definedas a surrounding vertex. Since the calibration jig 402 has the shape ofthe regular hexagonal pyramid, for example, an arbitrary surroundingvertex is designated as a vertex 601, and vertex numbers may be assignedcounterclockwise. Assume that detection positions of vertices 600 to 606on the image coordinate system are u_(J0) to u_(J6) as a result ofdetection of the vertices. Note that a position of the top vertex 600 ofthe calibration base 401 on the jig coordinate system is set to be a jigcoordinate system origin X_(J0)=[0, 0, 0]^(T).

The direction of a jig axis having the origin as the center isarbitrarily defined, and the positions of the vertices 601 to 606 on thejig coordinate system are calculated from X_(J0) based on the physicalrelative positions from the top vertex 600 and are set as positionvectors X_(J1) to X_(J6).

The relationship of the respective coordinate systems will be specifiedbelow.

Assume that an arbitrary point X=[X, Y, Z]^(T) on the jig coordinatesystem is transformed to a point x=[x, y, z]^(T) on a normalized imagecoordinate system by the coordinate transform based on a projectionmatrix A. At this time, a corresponding point u=[u, v]^(T) of X on theimage coordinate system is expressed by:

$\begin{matrix}{u^{\prime} = {{\frac{1}{2} \times} = {\frac{1}{z}{AX}^{\prime}}}} & (1)\end{matrix}$

where u′ and X′ are homogeneous coordinates of u and X to have a scalefactor=1, and are respectively expressed by u′=[u, v, 1]^(T) and X′=[X,Y, Z, 1]^(T). The projection matrix A is a 3×4 matrix, and each row isexpressed by the transposition of four-dimensional column vectors andcan be described by:

$\begin{matrix}{A = \begin{bmatrix}a_{1}^{T} \\a_{2}^{T} \\a_{3}^{T}\end{bmatrix}} & (2)\end{matrix}$

Homogeneous coordinates u′_(k)=[u_(k), v_(k), 1] associated with imagecoordinates u_(Ji)=[u_(Ji), v_(Ji)] associated with the i-th vertex inthe vertex detection results of the calibration jig 402 are expressed,from equation (1), by:

$\begin{matrix}{u_{Ji}^{\prime} = {{\frac{1}{z_{Ji}}{AX}_{Ji}^{\prime}} = {\frac{1}{z_{Ji}}\begin{bmatrix}{a_{1} \cdot X_{Ji}^{\prime}} \\{a_{2} \cdot X_{Ji}^{\prime}} \\{a_{3} \cdot X_{Ji}^{\prime}}\end{bmatrix}}}} & (3)\end{matrix}$

where z_(Ji) is a z-coordinate of the i-th vertex on the normalizedimage coordinate system, and X′_(Ji) represents homogeneous coordinatesof a coordinate value X_(Ji)=[X_(Ji), Y_(Ji), Z_(Ji)] of the i-th vertexon the jig coordinate system, which are expressed by X′_(Ji)=[X_(Ji),Y_(Ji), Z_(Ji), 1].

Since u′_(Ji) represents the homogeneous coordinates, it is equivalentwhen the entire equation is divided by a scale factor a₃·X′_(Ji) whileignoring z_(Ji). Therefore, respective elements of u_(Ji) can be writtenas:

$\begin{matrix}\left\{ \begin{matrix}{u_{Ji} = \frac{a_{1} \cdot X_{Ji}^{\prime}}{a_{3} \cdot X_{Ji}^{\prime}}} \\{v_{Ji} = \frac{a_{2} \cdot X_{Ji}^{\prime}}{a_{3} \cdot X_{Ji}^{\prime}}}\end{matrix} \right. & (4)\end{matrix}$

By modifying these equations, we have:

$\begin{matrix}\left\{ \begin{matrix}{a_{1} \cdot X_{Ji}^{\prime}} & {{{- u_{Ji}}{a_{3} \cdot X_{Ji}^{\prime}}} = 0} \\{a_{2} \cdot X_{Ji}^{\prime}} & {{{- v_{Ji}}{a_{3} \cdot X_{Ji}^{\prime}}} = 0}\end{matrix} \right. & (5)\end{matrix}$

Hence, these elements can be described using a matrix in the followingformat:

$\begin{matrix}{{\begin{bmatrix}X_{Ji}^{\prime \; T} & 0^{T} & {{- u_{Ji}}X_{Ji}^{\prime \; T}} \\0^{T} & X_{Ji}^{\prime \; T} & {{- v_{Ji}}X_{Ji}^{\prime \; T}}\end{bmatrix}\begin{bmatrix}a_{1} \\a_{2} \\a_{3}\end{bmatrix}} = 0} & (6)\end{matrix}$

for 0=[0, 0, 0]^(T). By equating a to:

$\begin{matrix}{a = \begin{bmatrix}a_{1} \\a_{2} \\a_{3}\end{bmatrix}} & (7)\end{matrix}$

and for all the jig vertices, equating P to:

$\begin{matrix}{P = \begin{bmatrix}X_{J\; 0}^{\prime \; T} & 0^{T} & {{- u_{J\; 0}}X_{J\; 0}^{\prime \; T}} \\0^{T} & X_{J\; 0}^{\prime \; T} & {{- v_{J\; 0}}X_{J\; 0}^{\prime \; T}} \\\; & \vdots & \; \\X_{J\; 6}^{\prime \; T} & 0^{T} & {{- u_{J\; 6}}X_{J\; 6}^{\prime \; T}} \\0^{T} & X_{J\; 6}^{\prime \; T} & {{- v_{J\; 6}}X_{J\; 6}^{\prime \; T}}\end{bmatrix}} & (8)\end{matrix}$

An equation of restraint condition:

Pa=0  (9)

is obtained. Since the number of vertices of the calibration jig 402 is7, an optimal approximate solution of a can be obtained by minimizing|Pa|² using the method of least squares, and a can be calculated as aneigenvector corresponding to a minimum eigenvalue of P^(T)P. From thisresult, the projection matrix A is estimated.

In step S502 (calibration movement process), the computer 140 supplies,to the robot controller 130, a robot coordinate system instructionΘ=[X_(R), Y_(R), Z_(R), θ_(RX), θ_(RY), θ_(RZ)] required to move therobot arm 111 to a predetermined position.

Note that X_(R), Y_(R), and Z_(R) are positions of the robot arm 111 onthe robot coordinate system, and θ_(RX), θ_(RY), and θ_(RZ) are rotationangles on the robot coordinate system.

The robot controller 130 transmits the instruction received from thecomputer 140 to the robot arm 111, and the robot arm 111 moves to thepredetermined position.

In step S503 (calibration data acquisition process), after movement ofthe robot arm 111, the computer 140 supplies a slit light emissioninstruction to the slit laser projector 113 and an image sensinginstruction to the camera 120. Then, image data is acquired in a statein which the calibration jig 402 and hand position detection member 114are irradiated with the slit light (FIG. 7).

In step S504 (slit light state calculation process), an equation ofplane of the light-section plane 200 on the jig coordinate system iscalculated from a light-section line 701 produced by the slit lightstriking on the calibration jig 402.

Bright spots of the light-section line are detected from the image data,and an equation of straight line on the image coordinate system can beobtained from a set of bright spots by, for example, the method of leastsquares. Since the image coordinate values u_(J0) to u_(J6) associatedwith the respective vertices of the calibration jig 402 are given, it iseasy to calculate intersections between sides defined by pairs ofvertices on the calibration jig 402 and the light-section line 701.

Assume that an intersection between a side defined by two vertices i andj of the calibration jig 402 and the light-section line 701 is obtainedas u_(L) on the image coordinate system. With respect to coordinatesu_(Ji) and u_(Jj) of vertices i and j on the image coordinate system,u_(L) is an internally dividing point of these two points.

Let u′_(Ji), u′_(Jj), and u′_(L) be corresponding points of these pointson the normalized image coordinate system, and X′_(Ji), X′_(Jj), andX′_(L) be homogeneous coordinate expressions of corresponding pointsX_(Ji), X_(Jj), and X_(L) on the jig coordinate system. Assuming thatX′_(L) is a point which internally divides X′_(Ji) and X′_(Jj), from theresult of the projective transform, we have:

u′ _(L) =AX′ _(L) =tAX′ _(Ji)+(1−t)AX _(Jj) =tu′ _(Ji)+(1−t)u′_(Jj)  (10)

If respective points on the normalized image coordinate system aredescribed as u′_(L)=[x_(L), y_(L), z_(L)], u′_(Ji)=[x_(Ji), y_(Ji),z_(Ji)], and u′_(Jj)=[x_(Jj), y_(Jj), z_(Jj)], equation (10) can bewritten for respective elements as:

$\begin{matrix}\left\{ \begin{matrix}{x_{L} = {{tx}_{J\; i} + {\left( {1 - t} \right)x_{Jj}}}} \\{y_{L} = {{ty}_{J\; i} + {\left( {1 - t} \right)y_{Jj}}}} \\{z_{L} = {{tz}_{J\; i} + {\left( {1 - t} \right)z_{Jj}}}}\end{matrix} \right. & (11)\end{matrix}$

Upon examining division of the image coordinates of u_(L)=[u_(L),v_(L)]^(T) by the scale factor of the homogeneous coordinates, we have:

$\begin{matrix}\left\{ \begin{matrix}{u_{L} = \frac{{tx}_{Ji} + {\left( {1 - t} \right)x_{Jj}}}{z_{L}}} \\{v_{L} = \frac{{ty}_{Ji} + {\left( {1 - t} \right)y_{Jj}}}{z_{L}}}\end{matrix} \right. & (12)\end{matrix}$

By eliminating zL from these equations, and solving them for t, we have:

$\begin{matrix}{t = \frac{{v_{L}x_{Jj}} - {u_{L}y_{Jj}}}{{u_{L}\left( {y_{Ji} - y_{Jj}} \right)} - {v_{L}\left( {y_{Ji} - y_{Jj}} \right)}}} & (13)\end{matrix}$

In this way, the corresponding point X_(L) of the intersection u_(L) onthe jig coordinate system is calculated.

If at least three intersections between the light-section line 701 andthe edges of the calibration jig 402 can be obtained, an equation ofplane of the light-section plane 200 on the jig coordinate system can beobtained. Note that when four or more intersections are obtained, asshown in FIG. 7, an equation of approximate plane can be estimated by,for example, the method of least squares.

Letting n be a normal vector of the equation of plane obtained in thisway, a vector equation of the plane is expressed, using an arbitrarypoint on the plane, for example, the previously measured point X_(L),by:

n ^(T)(X−X _(L))=0  (14)

If the homogeneous coordinates of a point X are expressed by X′=[X, Y,Z, 1], equation (14) can be rewritten by equating n′ to n′=[n^(T),−n^(T)X_(L)] as:

n′^(T)X′=0  (15)

where n′ is the Plücker coordinate expression of the equation of planeand may be normalized as follows since it is scale-invariable.

$\begin{matrix}{{\hat{n}}^{\prime} = \frac{n^{\prime}}{n^{\prime}}} & (16)\end{matrix}$

As a result, the equation of plane of the normalized light-section plane200 is obtained as:

{circumflex over (n)}′^(T)X′=0  (17)

A point x=[x, y, z] on the normalized image coordinate system, which isobtained by the projective transform of homogeneous coordinates X′ of acertain point X on this plane using the matrix A corresponds to thehomogeneous coordinates of an observation point u=[u, v] on thelight-section line obtained on the image. Hence, this point can beexpressed as x=[wu, wv, w] by rewriting z by a scale factor w. At thistime, in association with a projective transform formula:

x=AX′  (18)

by equating X′ to:

X′=[X^(T),1]^(T), a_(i)=[â_(i) ^(T),b_(i)]^(T) (i=1,2,3)

x can be written as:

$\begin{matrix}{\begin{bmatrix}{wu} \\{wv} \\w\end{bmatrix} = \begin{bmatrix}{{{\hat{a}}_{1}^{T}X} + b_{1}} \\{{{\hat{a}}_{2}^{T}X} + b_{2}} \\{{{\hat{a}}_{3}^{T}X} + b_{3}}\end{bmatrix}} & (19)\end{matrix}$

Hence, since 4-element simultaneous equations associated with X and wcan be obtained from equations (17) and (19), coordinates X on the jigcoordinate system corresponding to the observation position u on thelight-section line 701 on the image data can be obtained.

In step S505 (work unit state calculation process), the state of therobot arm 111 on the jig coordinate system is calculated from thelight-section line 201 striking on the hand position detection member114. Since the hand position detection member 114 and slit laserprojector 113 are arranged so that their relative positions remainunchanged, the light-section line 201 generated on the hand positiondetection member 114 is always generated at the same position on thehand position detection member 114.

In this case, since the hand position detection member 114 has atriangular sectional shape, the light-section line 201 on the handposition detection member 114 detected on the image can be obtained astwo sides of a triangle on the jig coordinate system from equations (17)and (19).

For example, letting n_(R) be a normal vector to a plane including theobtained triangle and X_(R) be the barycentric position, instructions Θon the robot coordinate system are decided to have a one-to-onecorrespondence with observation values Ω=[n_(R), X_(R)] of the positionand orientation of the robot arm 111.

In step S507 (calibration value calculation process), respectivetransform functions are generated based on the correspondence associatedwith the instructions on the robot coordinate system, light-sectionplanes, and observation positions and orientations of the robot arm 111,which are acquired at a plurality of positions.

Generation of a function required to estimate a light-section planecorresponding to an instruction on the robot coordinate system will bedescribed first. Assuming that a robot coordinate system instructionsupplied to obtain i-th calibration data is Θ_(i), and the Plückercoordinates of the light-section plane obtained at that time are{circumflex over (n)}′_(i), a plane equation estimation functionf(Θ)={circumflex over (n)}′ required to estimate the Plücker coordinates{circumflex over (n)}′_(i) of the light-section plane obtained uponinputting an arbitrary instruction Θ is generated. Using data setsT={Θ_(i)} for ∀i and N={{circumflex over (n)}′_(i)} for ∀i as samplepairs, their transform functions may be designed to obtain approximatesolutions of the plane equation estimation function f using the methodof least squares by setting nonlinear function models based onpolynomial regression.

When the work range of the robot arm 111 is wide, and low-order modelsbased on polynomial regression cannot be set, approximate solutions ofthe plane equation estimation function f may be obtained by, forexample, back-propagation learning using a multilayer perceptron.

Generation of a function required to estimate the value on the robotcoordinate system corresponding to the observation values of the robotarm 111 will be described below. Assume that a robot coordinate systeminstruction supplied to obtain the i-th calibration data is Θ_(i), andthe observation values of the robot arm 111 obtained at that time areΩ_(i). At this time, a robot coordinate estimation function g(Ω)=Θrequired to estimate a robot coordinate system instruction Θ obtainedupon inputting arbitrary Ω is generated.

Using data sets T={Θ_(i)} for ∀i and G={Ω_(i)} for ∀i as sample pairs,approximate solutions of the robot coordinate estimation function g areobtained in the same manner as in generation of the plane equationestimation function f in the calibration value calculation process (stepS507).

Calibration parameters of the functions f and g obtained by theaforementioned processes are stored in a storage device of the computer140 via the calibration value storage unit 305, thus ending thecalibration processing.

<5. Work Processing in Work System>

The sequence of the work processing for executing a predetermined workby measuring the 3D information of a target object using the calibrationparameters calculated by the aforementioned calibration method will bedescribed below.

<5.1 Overview of Work Processing>

An overview of the work processing in the work system will be describedfirst. FIG. 8 is a flowchart showing the sequence of the work processingin the work system 100.

In step S801 (measurement movement process), the measurement valuecalculation unit 302 sends a moving instruction to the work control unit130, which sends a control signal based on that instruction to the workunit 110. In response to this control signal, the work unit 110 moves toa position where a target object is irradiated with slit light.

In step S802 (measurement data acquisition process), the measurementvalue calculation unit 302 sends an image sensing start instruction tothe measurement control unit 301, which sends control signals to theimage sensing unit 120 and light projection unit 113. The image sensingunit 120 starts image sensing based on the control signal from themeasurement control unit 301, and the light projection unit 113 startsemission of slit light based on the control signal from the measurementcontrol unit 301. Note that image data acquired by the image sensingunit 120 is stored via the image data storage unit 303.

In step S803 (slit light state estimation process), the state of slitlight corresponding to the instruction sent to the work control unit 130is estimated using the correspondence function obtained based on thecalibration parameters stored via the calibration value storage unit 305(first calculation unit).

In step S804 (target object measurement value calculation process), themeasurement value calculation unit 302 detects a light-section line onthe target object in the image data using the image data stored via theimage data storage unit 303. Then, the measurement value calculationunit 302 calculates the measurement value of the target object on thejig coordinate system based on the state of the slit light obtained instep S803 (second calculation unit).

In step S805 (work target state calculation process), the state of thework unit 110 on the jig coordinate system in desired work contents iscalculated based on the target object measurement value obtained in stepS804.

In step S806 (work unit instruction calculation process), an instructionto be sent to the work control unit 130 is generated based on the stateof the work unit 110 on the jig coordinate system, which is calculatedusing the correspondence function obtained based on the calibrationparameters stored via the calibration value storage unit 305.

In step S807, the instruction obtained in step S806 is transmitted tothe work control unit 130, which sends a control signal based on theinstruction to the work unit 110. The work unit 110 operates based onthe control signal, thus executing a desired work.

<5.2 Details of Work Processing>

Details of the work processing shown in FIG. 8 will be described below.

In step S801 (measurement movement process), the robot arm 111 is movedto a predetermined position. In step S802 (measurement data acquisitionprocess), the slit laser projector 113 emits slit light, and the camera120 acquires image data.

In step S803 (slit light state estimation process), the Plückercoordinates {circumflex over (n)}′ of a light-section plane to beobtained in response to an instruction Θ supplied to the robot arm 111in step S802 is estimated using the plane equation estimation function fobtained at the time of calibration.

In step S804 (target object measurement value calculation process), thelight-section line on the target object 160 is detected, and {circumflexover (n)}′ obtained in step S803 (slit light state estimation process)is substituted in equations (17) and (19) to calculate the coordinatevalues of respective bright spot positions on the light-section line onthe jig coordinate system.

In step S805 (work target state calculation process), the position andorientation of the robot arm 111 on the jig coordinate system, which arerequired to attain a desired work, are calculated based on themeasurement result of the target object 160 obtained in step S804(target object measurement value calculation process).

By detecting the light-section line on the hand position detectionmember 114 first, the positions on the jig coordinate system associatedwith bright spots on the light-section line generated on the handposition detection member 114 are calculated in the same sequence as instep S804 (target object measurement value calculation process).

Based on the calculation result, the values Ω=[n_(R), X_(R)] on the jigcoordinate system of the normal vector and barycenter of a triangledefined by the light-section line on the hand position detection member114 are calculated in the same sequence as in step S505 (work unit statecalculation process).

The required target state depends on the work contents. A case will beexplained below wherein the target object is gripped. Based on therelative positional relationship between the measurement result of thetarget object 160 and that of the hand position detection member 114, ifthe robot arm 111 is required to move by ΔΩ=[Δn_(R), ΔX_(R)] on the jigcoordinate system, the target state can be obtained as:

{circumflex over (Ω)}=[n _(R) +Δn _(R) ,X _(R) +ΔX _(R)]

In step S806 (work unit instruction calculation process), the targetstate {circumflex over (Ω)} calculated in step S805 (work target statecalculation process) is transformed onto the robot coordinate system asan instruction to be supplied to the robot arm 111. By giving, as aninput value, {circumflex over (Ω)} to the coordinate estimation functiong obtained at the time of calibration, the corresponding coordinatevalue {circumflex over (Θ)} on the robot coordinate system can beobtained.

In step S807 (work process), the instruction {circumflex over (Θ)} tothe robot arm 111 obtained in step S806 (work unit instructioncalculation process) is transmitted to the robot controller 130 to movethe robot arm 111, thus executing a desired work.

As can be apparent from the above description, the work system 100according to this embodiment adopts the arrangement in which the camera120 is arranged at a position independent of the robot arm 111 withoutintegrating the camera 120, slit laser projector 113, and hand positiondetection member 114. Also, the work system 100 adopts the arrangementin which the slit laser projector 113 and hand position detection member114 are arranged on the robot arm 111.

With this arrangement, problems regarding occlusion and that regardingthe focal depth of beam light can be avoided. Since a large optic anglecan be assured, high measurement precision can be attained in the depthdirection, and the camera with a high resolution can be arrangedirrespective of the size of the robot arm.

Furthermore, by specifying the 3D information measurement method andcalibration method applied to this work system, this embodimentdemonstrates the feasibility of the measurement of the 3D information inthe work system.

Second Embodiment

The work processing in the first embodiment is premised on that 3Dinformation enough to execute a work in step S807 (work process) iscalculated in step S804.

However, 3D information required to uniquely estimate, for example, theposition and orientation of the target object 160 as the calculationresult in step S804 may not be obtained in some cases. Thus, thisembodiment will explain work processing premised on that 3D informationthat suffices to execute a work in step S807 (work process) cannot oftenbe obtained.

Note that the outer appearance arrangement, functional arrangement, andcalibration processing of the work system according to this embodimentare basically the same as those of the work system according to thefirst embodiment, and a description thereof will not be repeated.

FIG. 9 is a flowchart showing the sequence of work processing in thework system according to this embodiment.

In step S901 (measurement movement process) to step S904 (target objectmeasurement value calculation process), the same processes as in stepS801 (measurement movement process) to step S804 (target objectmeasurement value calculation process) in the work processing in thework system according to the first embodiment are executed.

It is determined in step S905 if 3D information enough to execute a workfor the target object is obtained as the measurement result in stepS904.

If it is determined in step S905 that sufficient 3D information is notobtained, the process returns to step S901 (measurement movementprocess). Then, the work unit 110 is moved to a position different fromthe current position to emit slit light from the different position,thus acquiring addition 3D information.

On the other hand, if it is determined in step S905 that sufficient 3Dinformation is obtained, the process advances to step S906. In step S906(work target state calculation process) to step S908 (work process), thesame processes as in step S805 (work target state calculation process)to step S807 (work process) in the work processing in the work systemaccording to the first embodiment are executed.

As can be seen from the above description, this embodiment can also copewith a case in which 3D information enough to execute a work in the workprocess is not obtained, while receiving the effects obtained by thefirst embodiment.

Third Embodiment

In the first embodiment, the irradiation angle of slit light to beemitted is fixed, and the shape of the hand position detection member isdefined to uniquely determine the position and orientation of the robotarm 111 based on the sensed light-section line 201.

However, the present invention is not limited to this. For example, theirradiation angle of slit light to be emitted by the slit laserprojector 113 may be variable so as to uniquely determine the positionand orientation of the robot arm 111. Details of this embodiment will bedescribed below.

<1. Outer Appearance Arrangement of Work Unit>

FIG. 10 is a view showing the outer appearance arrangement of a workunit of a work system according to this embodiment. Referring to FIG.10, reference numeral 1003 denotes a movable galvano mirror or polygonmirror, which serves as a light scanning unit. As shown in FIG. 10, thelight scanning unit 1003 is arranged on the front portion of the slitlaser projector 113 to change the irradiation angle of a light-sectionplane 1000 of slit light based on an instruction from the measurementcontrol unit 301.

Note that the light scanning unit 1003 generates K light-section linesper scan. Let α_(k) be a slit light irradiation angle from the originposition of the light scanning unit 1003, which is required to generatethe k-th light-section line of these K lines. K and α₁ to α_(k) arefixed values, which are defined in advance, and the same values are usedin the calibration processing and work processing.

Reference numeral 1004 denotes a hand position detection member. Notethat since K light-section lines are generated per scan on the handposition detection member 1004, the hand position detection member 1004need not have a complicated shape unlike in the first embodiment, and itmay be, for example, a flat plate, as shown in FIG. 10. When the handposition detection member 1004 has a flat plate shape, a light-sectionline 1001 forms one straight line.

<2. Measurement Principle of Target Object in Work System>

The calibration method required to measure 3D information by thelight-section method will be described below.

<2.1 Description of Calibration Jig>

A calibration jig used in calibration of the work system according tothis embodiment is the same as that used in calibration of the worksystem according to the first embodiment, and a description thereof willnot be repeated.

<2.2 Details of Calibration Processing>

The sequence of the calibration processing in the work system accordingto this embodiment will be described below. FIG. 11 is a flowchartshowing the sequence of the calibration processing in the work system.

In step S1101 (jig coordinate system setting process) and step S1102(calibration movement process), the same processes as in step S501 (jigcoordinate system setting process) and step S502 (calibration movementprocess) described in the first embodiment are executed.

Note that in step S1102, after a counter value k used to count thenumber of times of change of the slit light irradiation angle is resetto 1, the process advances to step S1103 (irradiation angle changeprocess).

In step S1103 (irradiation angle change process), a slit lightirradiation angle in the light scanning unit 1003 is set to be α_(k).

In step S1104 (calibration data acquisition process) and step S1105(slit light state calculation process), the same processes as in stepS503 (calibration data acquisition process) and step S504 (slit lightstate calculation process) described in the first embodiment areexecuted. With these processes, an equation of plane of thelight-section plane 1000 on the jib coordinate system is obtained.

In step S1106 (work unit measurement process), the coordinate value onthe jig coordinate system associated with light-section line obtained atthe slit light irradiation angle α_(k) on the hand position detectionmember 1004 is calculated using equations (17) and (19).

It is checked in step S1107 if the counter value k=K. If it isdetermined in step S1107 that the counter value k<K, the counter value kis incremented, and the process then returns to step S1103.

On the other hand, if it is determined in step S1107 that the countervalue k=K, the process advances to step S1108.

In step S1108 (work unit state calculation process), the position andorientation of the robot arm 111 on the jig coordinate system arecalculated based on the coordinate values on the jig coordinate systemassociated with the light-section lines on the hand position detectionmember 1004 obtained at the slit light irradiation angles α₁ to α_(k).

If the hand position detection member 1004 is, for example, a flatplate, as shown in FIG. 10, the light-section lines on the hand positiondetection member 1004 obtained at the slit light irradiation angles α₁to α_(k) are as shown in FIG. 12.

That is, K line segments on a single plane are formed on the jigcoordinate system, and their normal direction n_(R) can be obtained by,for example, the method of least squares. Also, the barycentric positionof a region bounded by edges 1201 and 1202 of the hand positiondetection member 1004 and light-section lines 1203 and 1204 as those attwo ends is calculated as X_(R). Then, instructions Θ on the robotcoordinate system are decided to have one-to-one correspondence withobservation values Ω=[n_(R), X_(R)] of the position and orientation ofthe robot arm 111.

It is checked in step S1109 if the number of times of data acquisitionreaches a predetermined value. If it is determined in step S1109 thatthe number of times of data acquisition does not reach the predeterminedvalue, the process returns to step S1102, and the robot arm 111 is movedto another position to continue acquisition of calibration data.

On the other hand, if it is determined in step S1109 that the number oftimes of data acquisition reaches the predetermined value, the processadvances to step S1110.

In step S1110 (calibration value calculation process), respectivetransform functions are generated based on the correspondence associatedwith instructions on the robot coordinate system, light-section planes,and the observation positions and orientations of the robot arm 111acquired at the plurality of positions and the plurality of slit lightirradiation angles.

Assume that a robot coordinate system instruction supplied to obtaincalibration data at the i-th calibration position is Θ_(i), and Plückercoordinates of light-section planes respectively obtained at the slitlight irradiation angles α₁ to α_(k) at that time are {circumflex over(n)}′_(ik). At this time, a plane equation estimation function f(Θ,α)={circumflex over (n)}′ required to estimate Plücker coordinates{circumflex over (n)}′ of a light-section plane obtained upon inputtingarbitrary Θ and α is generated.

Using data sets T={[Θ_(i)α_(k)]} for ∀i, k and N={{circumflex over(n)}′_(ik)} for ∀i, k as sample pairs, their transform functions may bedesigned to obtain approximate solutions of the plane equationestimation function f using the method of least squares by settingnonlinear function models based on polynomial regression. When the workrange of the robot arm 111 is wide, and low-order models based onpolynomial regression cannot be set, approximate solutions of the planeequation estimation function f may be obtained by, for example,back-propagation learning using a multilayer perceptron.

Assume that a robot coordinate system instruction supplied to obtain thei-th calibration data is Θ_(i), and the observation values of the robotarm 111 obtained by irradiating slit light beams at the slit lightirradiation angles α₁ to α_(k) at that time are Ω_(i). At this time, arobot coordinate estimation function g(Ω)=Θ required to estimate a robotcoordinate system instruction Θ obtained upon inputting arbitrary Ω isgenerated.

Using data sets T={Θ_(i)} for ∀i and G={Ω_(i)} for ∀i as sample pairs,approximate solutions of the robot coordinate estimation function g areobtained in the same manner as in generation of the plane equationestimation function f in step S1110 (calibration value calculationprocess).

Calibration parameters of the functions f and g obtained by theaforementioned processes are stored in a storage device of the computer140 via the calibration value storage unit 305.

<3. Details of Work Processing>

Details of the work processing in the work system according to thisembodiment will be described below. FIG. 13 is a flowchart showing thesequence of the work processing in the work system according to thisembodiment.

In step S1301 (measurement movement process), the same process as instep S801 (measurement movement process) in the first embodiment isexecuted, and the robot arm 111 is moved. Note that in step S1301, afterthe counter value k used to count the number of times of change of theslit light irradiation angle is reset to 1, the process advances to stepS1302 (irradiation angle change process).

In step S1302 (irradiation angle change process), a slit lightirradiation angle in the galvano mirror 1003 is set to be α_(k).

In step S1303 (measurement data acquisition process) to step S1305(target object measurement value calculation process), the sameprocesses as in the first embodiment are executed to calculate thecoordinate value on the jig coordinate system of a light-section line onthe target object 160.

It is checked in step S1306 if the counter value k=K. If it isdetermined in step S1306 that the counter value k<K, the counter value kis incremented, and the process returns to step S1302.

On the other hand, if it is determined in step S1306 that the countervalue k=K, the process advances to step S1307.

It is determined in step S1307 if 3D information that suffices toexecute a work for the target object is obtained as the measurementresult. For example, if it is determined that 3D information required touniquely estimate the position and orientation of the target object 160is not obtained, it is determined that sufficient 3D information is notobtained, and the process returns to step S1301. Then, the robot arm 111is moved to a position different from the current position, andadditional 3D information is acquired by irradiating slit light from thedifferent position.

On the other hand, if it is determined in step S1307 that 3D informationthat suffices to execute a work for the target object is obtained as themeasurement result, the process advances to step S1308.

In step S1308 (work target state calculation process), the same processas in the work unit state calculation process (step S1108) at the timeof calibration is executed to calculate the position and orientation ofthe robot arm 111.

In step S1309 (work unit instruction calculation process) and step S1310(work process), the same processes as in the first embodiment areexecuted.

As can be seen from the above description, this embodiment adopts thearrangement in which the irradiation angle of slit light emitted by theslit laser projector 113 is variable. Then, the position and orientationof the robot arm 111 can be uniquely calculated irrespective of theshape of the hand position detection member.

Fourth Embodiment

In the first, second, and third embodiments, the light projection unit113 projects slit light.

However, the present invention is not limited to this. For example, anarrangement in which a pattern light projector used to attain spatialcoding is used as the light projection unit 113 may be adopted.

Note that spatial coding is a method of coding a space by irradiatingand sensing a binary pattern light combination based on a densitypattern emitted by a light projector, while changing it in synchronismwith a camera, and obtaining distance information by a triangulationwith an associated light projection position from the coded space.Details of this embodiment will be described below.

FIG. 14 is a view showing the outer appearance arrangement of a workunit of a work system according to this embodiment. Referring to FIG.14, reference numeral 1011 denotes a pattern light projector whichprojects pattern light while switching a pattern in synchronism with thecamera 120 based on an instruction from the measurement control unit301.

As shown in FIG. 15, a 1-byte information amount is included at anarbitrary position in a sensed image using binary pattern lightprojected once by the pattern light projector 1011. In this case, forexample, as shown in FIG. 16, N types of pattern light based on changesto double the frequencies of periodic patterns in the vertical directionare given, and images of respective light projection patterns aresensed. Then, by binarizing luminance changes at an arbitrary positionon an image and time-serially arranging them, N-byte binary codeinformation is given.

At this time, a point group on an image having an identical binary codeis located on an identical irradiation plane from the pattern lightprojector 1011, as shown in the example of FIG. 15, since it undergoesan identical pattern light change. For this reason, when theaforementioned pattern change is used, points on 2^(N) planes can bediscriminated on an image.

By examining these 2^(N) planes as K light-section planes, a problemequivalent to the third embodiment is to be solved, and calibration dataacquisition and 3D information acquisition of a target object at thetime of a work can be attained by the same method as in the thirdembodiment.

As can be seen from the above description, this embodiment adopts thearrangement in which the pattern light projector which projects patternlight is used as the light projection unit 113.

As a result, even in spatial coding based on pattern light, 3Dinformation can be measured and calibrated by the same method as in thethird embodiment.

Other Embodiments

Aspects of the present invention can also be realized by a computer of asystem or apparatus (or devices such as a CPU or MPU) that reads out andexecutes a program recorded on a memory device to perform the functionsof the above-described embodiment(s), and by a method, the steps ofwhich are performed by a computer of a system or apparatus by, forexample, reading out and executing a program recorded on a memory deviceto perform the functions of the above-described embodiment(s). For thispurpose, the program is provided to the computer for example via anetwork or from a recording medium of various types serving as thememory device (e.g., computer-readable medium).

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application No.2008-301714 filed Nov. 26, 2008, which is hereby incorporated byreference herein in its entirety.

1. A work system comprising: an arm mechanism, a position and anorientation of which are configured to be changed so as to execute apredetermined work for an object to be processed placed on a work area;a member arranged at a distal end portion of said arm mechanism; anirradiation unit configured to form a light projection pattern on saidmember and on the object to be processed by irradiating said member andthe object to be processed with pattern light, said irradiation unitbeing arranged on said arm mechanism to have a constant relativeposition with respect to said member; an image sensing unit configuredto sense an image of the object to be processed, said image sensing unitbeing fixed at a position independent of said arm mechanism; a firstcalculation unit configured to calculate a position and an orientationof the distal end portion of said arm mechanism and an irradiation planeof the pattern light based on the light projection pattern formed onsaid member included in image data of the object to be processed whichis obtained by image sensing by said image sensing unit; and a secondcalculation unit configured to calculate a position and an orientationof the object to be processed based on the calculated irradiation planeand the light projection pattern formed on the object to be processedincluded in the image data of the object to be processed which isobtained by image sensing by said image sensing unit.
 2. The systemaccording to claim 1, further comprising a control unit configured tocontrol an operation of said arm mechanism based on the position and theorientation of the distal end portion of said arm mechanism calculatedby said first calculation unit, and the position and the orientation ofthe object to be processed calculated by said second calculation unit.3. The system according to claim 1, wherein said irradiation unit is aslit light projector which irradiates said member and the object to beprocessed with slit light as the pattern light, and forms alight-section line as the light projection pattern on said member and onthe object to be processed.
 4. The system according to claim 3, whereinsaid member has a shape with which the light-section line formed on saidmember by the slit light irradiated by said irradiation unit becomes alight-section line defined by at least two straight lines.
 5. The systemaccording to claim 3, wherein said irradiation unit is configured tochange an irradiation angle with respect to said member.
 6. The systemaccording to claim 5, wherein said member has a shape with which thelight-section line formed on said member by the slit light irradiated bysaid irradiation unit becomes a light-section line defined by at leastone straight line.
 7. The system according to claim 6, wherein saidfirst calculation unit calculates the positions and the orientations ofthe distal end portion of said arm mechanism and the light-sectionplanes of the slit light based on a plurality of light-section linesformed by changing the irradiation angle with respect to said member. 8.The system according to claim 1, wherein said second calculation unitcalculates the position and the orientation of the object to beprocessed based on a relationship between the position and theorientation of said arm mechanism and the irradiation plane of thepattern light, which are calculated based on a light projection patternformed on an object having given dimensions by irradiating the objecthaving the given dimensions with the pattern light by said irradiationunit when said arm mechanism is located at a predetermined position andorientation.
 9. The system according to claim 1, wherein saidirradiation unit is a pattern light projector which irradiates saidmember and the object to be processed with pattern light whichtime-serially changes, and time-serially forms light projection patternson said member and on the object to be processed.
 10. An informationprocessing method in a work system which comprises: an arm mechanism, aposition and an orientation of which are configured to be changed so asto execute a predetermined work for an object to be processed placed ona work area; a member arranged at a distal end portion of the armmechanism; an irradiation unit configured to form a light projectionpattern on the member and on the object to be processed by irradiatingthe member and the object to be processed with pattern light, theirradiation unit being arranged on the arm mechanism to have a constantrelative position to the member; and an image sensing unit configured tosense an image of the object to be processed, the image sensing unitbeing fixed at a position independent of the arm mechanism, the methodcomprising the steps of: calculating a position and an orientation ofthe distal end portion of the arm mechanism and an irradiation plane ofthe pattern light based on the light projection pattern formed on themember included in image data of the object to be processed which isobtained by image sensing by the image sensing unit; and calculating aposition and an orientation of the object to be processed based on thecalculated irradiation plane and the light projection pattern formed onthe object to be processed included in the image data of the object tobe processed which is obtained by image sensing by the image sensingunit.
 11. A computer-readable storage medium storing a program formaking a computer execute an information processing method according toclaim 10.